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ABSTRACT 



This paper describes a project in the Department of 
Economics at Southeast Missouri State University undertaken to assess the 
actual and perceived impact of the World Wide Web upon student learning. Pre- 
and post -surveys were administered to students in "user" (i.e., instructors 
who incorporate the Web at least eight times in class and require students to 
use it for assignments) and "non-user" (i.e., instructors who incorporate the 
Web less than three times and do not require its use for assignments) 
sections of freshman and sophomore economics classes. These surveys measured 
student perceptions regarding: how the use of the Web in class or^for 
assignments enhances learning; how the use of the Web makes the class more 
interesting; whether teachers should require Web assignments; whether the 
class is boring and irrelevant; convenience of obtaining access to the Web; 
and expected grade. Results indicate that the use of the Web improves 
students' attitudes toward the course. However, no evidence was found to 
support the claim that Web-based instruction improves actual or perceived 
student performance . (AEF) 
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Abstract 

The use of the World Wide Web for the delivery of course information is growing at a rapid rate. However, little 
research has been done to assess the impact of Web-based instruction upon student attitude and performance. This 
paper describes a project undertaken to assess the actual and perceived impact of the Web upon student learning. 
Results indicate that the use of the Web improves students' attitude toward the course. However, no evidence was 
found to support the claim that Web-based instruction actually improves actual or perceived student performance. 

1T0P1 

Introduction 

The use of the World Wide Web for the delivery of course information is growing at a rapid rate. The degree of 
utilization ranges from simple on-line course syllabi to detailed on-line courses designed for distance learning. 

Some research has been done to determine student attitude toward the Web, such as Lehman and Lehman ( 1996) 
and Takle and Taber ( 1996). Less has been done in the area of assessing impact of Web-based instruction upon 
student learning. Universities are clearly interested in determining the impact of technology upon student learning. 

A story in the February 19, 1998 on-line issue of the Chronicle of Higher Education describes a survey developed by a 
non-profit organization. Teaching Learning Technology Group (affiliated with AAHE), that enables universities to 
determine technology's impact upon student learning 

The purpose of this paper is to present the initial results of a project undertaken in the Department of Economics at 
Southeast Missouri State University designed to determine the impact of Web-based instruction upon student 
attitude and performance in freshman and sophomore level economics courses. In its Strategic Plan developed in 
September 1995, the department cited "increased usage of Internet resources" for classroom instruction as one of its 
objectives. The utilization of technology as an integral component of instruction is also been stated as a priority of 
the Harrison College of Business, in which the Economics department is housed, and of the University. 

Competency in the use of the Internet is a necessary skill that all students should develop before graduation; ideally, 
this competence will be developed early, enabling them to use the Internet for information gathering throughout 
their college career. Many university students grew up with the Internet, and may respond positively to efforts to 
include this technology in the classroom. The technological facilities in the recently constructed Robert Dempster 
Hall allow faculty to incorporate the Internet directly into classroom activities. Labs throughout campus allow 
students to use the Internet to complete out-of-class assignments. 
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Purpose of the Project 

The purpose of this project was to increase student interest in 100 and 200 level economics courses, and reduce the 
percentage of Ds and Fs earned in these courses, by incorporating the Internet (more specifically, the World Wide 
Web) into economics courses. During the past five years, the percentage of low grades earned in EC 101 (Economic 
Problems and Policies), EC2 15 (Principles of Microeconomics), and EC225 (Principles of Macroeconomics) 
ranged from seventeen percent to almost thirty percent. Results from student evaluations and conversations with 
students cite low student interest as a problem. Increasing student interest may also lead to an increase in retention, 
which is a goal of the University. 

According to the plan of the project, participating faculty would have websites completed by Fall 1997 and would 
use the Web in courses during that semester. Web-based instructional material included on-line course syllabi, 
on-line tutorials, links to course related resources on the Web, and bulletin boards. Three instructors, teaching seven 
sections of lower level economics courses, were identified as "users, incorporating the Web at least eight times in 
class and requiring students to use it for at 

least one assignment. Four instructors, teaching eleven sections of lower level economics courses, were identified as 
"non-users," incorporating the Web less than three times during class and not requiring its use for out of class 
assignments. It is worth noting, however, that 49% of students in the "non-user" sections reported that they did 
utilize the World Wide Web at least one time in preparing assignments. 

rTOPl 

Survey Results 

The most extensive source of information regarding student attitudes toward Web usage in economics classes was 
obtained from two surveys that were distributed to all sections of EC 101, EC215 and EC225 during the Fall 1997 
semester. A pre-survey was distributed during the first week of class, and a post-survey during the last week of 
class. 269 students in the "user" classes completed the pre-survey, and 169 completed the post survey. 307 students 
in the "non-user" classes completed the pre-survey, and 239 complete the post-survey. 

Surveys were processed according to whether or not the instructor was a "user" of the Web. As tables 1 through 3 
show, results of the pre-survey show that there was no statistically significant difference between students in the 
"user" classes and "non-user" classes with respect to their initial opinions about the value of the World Wide Web 
for instructional purposes. 

On the pre-survey, students were asked to respond to the following statement: "Using the World Wide Web in class 
or for class assignments has enhanced my learning." As the information in the following table shows, there was 
little difference in responses between "user" classes and "non-user" classes. 

Table 1 



i 


Users 


Non-users 


Strongly Agree 


21% 


15% 


Agree 


44% 


46% 


Not sure 


28% 


27% 


Disagree 


5% 


7% 


Strongly Disagree 


2% 


4% 



In the user group, 65% of students agree or strongly agree with this statement, while in the non-user group, 61% of 
students agree or strongly agree with this statement. Testing the null hypothesis of no difference between 




5/17/99 9:26 AM 



The Effectiveness of the World Wide Web as an Instructional Tool 



http://www.mtsu.edu/~itconf/proceed98/rsummary.html 



percentages (H 0 : p u = P n h a :p u = p n ), a z value of .99 leads one to fail to reject the null hypothesis of no difference. Well 
over half of the students believed that the Web had enhanced their learning in other classes. 

On the pre-survey, students were asked to respond to the following statement: "Using the World Wide Web in class 
or for class assignments makes the class more interesting. " 



Table 2 





Users 


Non-users 


Strongly Agree 


21% 


15% 


Agree 


44% 


45% 


Not sure 


22% 


26% 


Disagree 


11% 


11% 


Strongly Disagree 


2 % 


3% 



In the user group, 65% of students agree or strongly agree with this statement, while in the non-user group, 60% of 
students agree or strongly agree with this statement. Testing the null hypothesis of no difference between 
percentages (H 0 : p u = P n h a :p u = p n ), a z-value of 1 .32 leads one to fail to reject the null hypothesis of no difference. 
Well over half of the students believed that the Web had made classes more interesting. 

On the pre-survey, students were asked to respond to the following statement: "Teachers should required World 
Wide Web assignments for their classes." 



Table 3 





Users 


Non-users : 

1 


| Strongly Agree 


9% 


1 

N© 

o x 

OO 


1 Agree 


26% 


25% 


1 Not sure 


29% 


33% 


! Disagree 


29% 


25% 


Strongly Disagree 


7% 


10% 



In the user group, 34% of students agree or strongly agree with this statement, while in the on-user group, 33% of 
students agree or strongly agree with this statement. Testing the null hypothesis of no difference between 
percentages (H 0 : p u = p n h a :p u = p n ), a z-value of .66 leads one to fail to reject the null hypothesis of no difference. 

Results of the post-survey, found in Tables 4 and 5, show that there was a statistically significant difference 
between students in the "user" classes and "non-user" classes with respect to their opinions about the value of the 
World Wide Web in their economics classes. Over three-quarters of the students in the "user" classes found the 
WWW to be a valuable instructional tool. 
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On the post-survey, students were asked to respond to the following statement: "Using the World Wide Web in my 
economics class has enhanced my learning." 



Table 4 



I 


Users 


Non-users 1 


Strongly Agree 


17% 


5% 


Agree 


63% 


27% 


Disagree 


12% 


8% 


| Strongly Disagree 


5% 


2% 


Not applicable 


2% 


58% 

1 



In the user group, 80% of students agree or strongly agree with this statement, while in the non-user group, 32% of 
students agree or strongly agree with this statement. Testing the null hypothesis of no difference between 
percentages (H 0 : p u = p n h a :p u = P n ), a z-value of 4.77 leads one to reject the null hypothesis of no difference at the .001 
level. There is a very high level of support for use of the WWW by students in the "user" classes, and, not 
surprisingly, this support is significantly higher than that in the "nonuser" classes. It is also important to note that 
well over half of the students in the "nonuser" classes who used the WWW on their own found that it enhanced their 
learning. (According to post-survey survey results, 49% of students in the "non-user" classes used the WWW at 
least once for assignments in their economics classes.) 

On the post-survey, students were asked to respond to the following statement: "Using the World Wide Web in my 
economics class has made my economics class more interesting. " 



Table 5 





Users 


Non-users 


Strongly Agree 


20% 


8% 


Agree 


63% 


22% 


Disagree 


12% 


12% 


Strongly Disagree 


4% 


2% 


Not applicable 


1% 


55% 



In the user group, 83% of students agree or strongly agree with this statement, while in the non-user group, 30% of 
students agree or strongly agree with this statement. Testing the null hypothesis of no difference between 
percentages (H 0 : p u = p n h a :p u = p n ), a z-value of 5.27 leads one to reject the null hypothesis of no difference at the .001 
level. Again, there is a very high level of support for use of the WWW by students in the "user" classes, and, not 
surprisingly, this support is significantly higher than that in the "non-user" classes. It is also important to note that 
well over half of the students in the "non-user" classes who used the WWW on their own found that it made the 
class more interesting. 

Results of the pre-survey (Table 6) show that there was no statistically significant difference between students in the 
"user" classes and "non-user" classes with respect to their initial opinion about whether or not they expect their 
economics class to be boring. However, there was a difference after students took their economics class (Table 7). 

On the pre-survey, students were asked to respond to the following statement: "I expect my economics class to be 
boring and irrelevant." 
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Table 6 





Users 


Non-users 


Strongly Agree 


3% 


2% 


Agree 


5 % 


8% 


Not sure 


41% 


40% 


Disagree 


43% 


40% 


Strongly Disagree 


9 % 


9% 



In the user group, 8% of students agree or strongly agree with this statement, while in the non-user group, 10% of 
students agree or strongly agree with this statement. Testing the null hypothesis of no difference between 
percentages (H 0 : p u = p n h a :P u = P n ), a z-value of .69 leads one to fail to reject the null hypothesis of no difference. 

On the post-survey, students were asked to respond to the following statement: "My Economics class has been 
boring and irrelevant." 



Table 7 





Users 


Non-users 


Strongly Agree 


3% ' 


10% 


Agree 


8% 


15% 


Not sure 


19% 


15% 


Disagree 


56% 


47% 


Strongly Disagree 


14% 


14% 



In the user group, 1 1% of students agree or strongly agree with this statement, while in the non-user group, 25% of 
students agree or strongly agree with this statement. Testing the null hypothesis of no difference between 
percentages (h 0 : p u = P n h a :p u = P n ), a z-value of-3.62 leads one to reject the null hypothesis of no difference at the .001 
level. A significantly larger percentage of students in the "non-user" classes found their economics class to be 
boring and irrelevant. Of course, it is not possible to attribute this difference solely to the use of the World Wide 
Web. There may be other differences in teaching style between the "user" and "non-user" instructors, and 
differences in learning styles between students in "user" and "non-user" classes, that account for this difference. 

Yet, it is an interesting result that bears further investigation. 

Did lack of access to computer facilities affect the responses of the two groups? Evidence from the pre- and 
post-surveys (Table 8) indicate that it did not. 

On both surveys, students were asked to respond to the following statement: "I can conveniently obtain access to 
the World Wide Web using my own computer or a computer in the student labs or dorms on campus." 

Table 8 
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User Pre-survey 


Non-users Pre-survey 


Users Post-survey 


Non-users Post-survey 


Strongly Agree 


31% 


29% 


45% 


49% 


Agree 


42% 


45% 


41% 


35% 


Not sure 


11% 


12% 


3% 


5% 


Disagree 


10% 


11% 


9% 


8% 


Strongly Disagree 


6% 


4% 


2% 


4% 



On the pre-survey, 73% of students in the "user" classes agreed or strongly agreed with this statement; the 
corresponding percentage in the "non-user" classes was 74%. On the post-survey, 86% of students in the "user" 
classes agreed or strongly agreed with this statement, while the percentage in the "non-user" classes was 84%. 
Clearly, there is no difference between the two groups on either survey. 

Did use of the World Wide Web have an impact on students' perception of their grades? On the pre-survey and 
post-survey, students were asked to respond to the following statement: "I expect my grade in economics this 
semester to be a(an):" As Table 9 shows, on the pre-survey, there was no real difference in student's expectations of 
their grades. 93% of students in the "user" classes expected to receive an A or a B, while 94% of students in the 
"non-user" classes expected to receive an A or a B. Likewise on the post-survey, 61% of students in the "user" 
classes expected an A or a B, while 63% of students in the "non-user" classes expected an A or a B. 



Table 9 





Users Pre-survey 


Non-users Pre-survey 


Users Pre-survey 


Non-users Pre-survey 


A ; 


47% 


53% 


25% 

i 


10% 


B 


46% | 


41% 


36% | 

l 


44% 


C 


6% 


6% 


34% 


33% 


D 


1% 


0% 


5% 


4% 


F 


0% 


0% 


0% 


1% 



On the pre-survey, there was no real difference in student's expectations of their grades. 93% of students in the 
"user" classes expected to receive an A or a B, while 94% of students in the "non-user" classes expected to receive 
an A or a B. Likewise on the post-survey, 61% of students in the "user" classes expected an A or a B, while 63% of 
students in the "non-user" classes expected an A or a B. 

TTOPl 

Results from Student Evaluation of Instruction (IDEA) 

All instructors in the Department of Economics used the IDEA student evaluation form in the Fall 1997 semester. 
There is a percentile score reported for all instructors regarding students' perception of their "improved attitude 
toward field." The weighted (according to number of students in class) average percentile for instructors using the 
WWW was 54, while the weighted average percentile for those instructors not utilizing the WWW was 30. Again, it 
is not possible to attribute this difference solely to use of the WWW, but it is a result that is worth further 
investigation. 
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Results From Actual Student Performance 

It is not possible to compare the grade distribution of "user" classes and "nonuser" classes for a variety of reasons, 
the most obvious being that most of the "non-user" classes were EC215, while the majority of "user" classes were 
EC 101. Grades tend to be somewhat lower in EC2 1 5 than in EC 101 . 

We might look at the grade distribution of individual instructors to determine if use of the WWW resulted in fewer 
Ds and Fs. Because of confidentiality issues, individual grade distributions are not reported here. However, an 
examination of the data indicate no significant change in the number of Ds and Fs after instructors incorporated the 
WWW into their classes. 

1TOP1 

Conclusions 

Tentative results of this study indicate the use of the World Wide Web in economics classes makes economics more 
interesting to students and, according to students, increases their learning of economics. Results from student 
evaluations of instruction tend to support the hypothesis that utilization of the WWW improves student attitude 
toward economics. However, students' expectations of grades and actual grades given to students do not support the 
hypothesis that this increased interest results in improved performance. 

The initial results of this study are interesting enough to warrant further investigation. The pre- and post- surveys 
are being distributed to all lower level economics classes in the Spring 1998 semester in order to increase the size of 
the sample. Further analysis will be conducted to separate the impact of WWW usage upon student attitude and 
performance from the impact of other instructional techniques. Also, differences in attitudes toward the WWW by 
gender, age, and year in school will be examined. 
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